Cursor mode switching

ABSTRACT

Methods and systems for processing input from an image-capture device for gesture-recognition. The method further includes computationally interpreting user gestures in accordance with a first mode of operation; analyzing the path of movement of an object to determine an intent of a user to change modes of operation; and, upon determining an intent of the user to change modes of operation, subsequently interpreting user gestures in accordance with the second mode of operation.

RELATED APPLICATION

This application is a continuation of U.S. Pat. Application No.17/896,619, titled: “CURSOR MODE SWITCHING”, filed Aug. 26, 2022(Attorney Docket No. ULTI 1047-10), which is a continuation of U.S. Pat.Application No. 17/542,990, titled: “CURSOR MODE SWITCHING”, filed Dec.6, 2021 (Attorney Docket No. ULTI 1047-9), which is a continuation ofU.S. Pat. Application No. 17/155,019, titled: “CURSOR MODE SWITCHING”,filed Jan. 21, 2021 (Attorney Docket No. ULTI 1047-8) issued as U.S.Pat. No. 11,194,404 on Dec. 7, 2022, which is a continuation of U.S.Pat. Application No. 16/660,528, titled: “CURSOR MODE SWITCHING”, filedOct. 22, 2019 (Attorney Docket No. ULTI 1047-7) issued as U.S. Pat. No.10,901,519 on Jan. 26, 2021, which is a continuation of U.S. Pat.Application No. 16/377,072, titled: “CURSOR MODE SWITCHING”, filed 5Apr. 2019 (Attorney Docket No. ULTI 1047-6) issued as U.S. Pat. No.10,459,530 on Oct. 29, 2019, which is a continuation of U.S. PatentApplication No. 15/933,199, titled: “CURSOR MODE SWITCHING”, filed 22Mar. 2018 (Attorney Docket No. ULTI 1047-5) issued as U.S. Pat. No.10,254,849 on Apr. 9, 2019, which is a continuation of U.S. Pat.Application No. 15/411,909, titled: “CURSOR MODE SWITCHING”, filed 20Jan. 2017, (Attorney Docket No. ULTI 1047-4) issued as U.S. Pat. No.9,927,880 on Mar. 27, 2018, which is a continuation of U.S. Pat.Application No. 15/213,899, titled: “CURSOR MODE SWITCHING”, filed 19Jul. 2016, (Attorney Docket No. ULTI 1047-3) issued as U.S. Pat. No.9,552,075 on Jan. 24, 2017, which is a continuation of U.S. Pat.Application No. 14/281,817, titled “CURSOR MODE SWITCHING”, filed 19 May2014 (Attorney Docket No. ULTI 1047-2) issued as U.S. Pat. No. 9,436,288on Sep. 6, 2016, which claims the benefit of U.S. Pat. Application No.61/824,691, titled “CURSOR MODE SWITCHING”, filed 17 May 2013 (AttorneyDocket No. LEAP 1047-1).

FIELD OF THE TECHNOLOGY DISCLOSED

The present disclosure relates generally to gesture recognition and, inparticular, to interpreting gesture-based user input.

BACKGROUND

The subject matter discussed in the background section should not beassumed to be prior art merely as a result of its mention in thebackground section. Similarly, a problem mentioned in the backgroundsection or associated with the subject matter of the background sectionshould not be assumed to have been previously recognized in the priorart. The subject matter in the background section merely representsdifferent approaches, which in and of themselves may also correspond toimplementations of the claimed technology.

Traditionally, users have interacted with electronic devices, such as acomputer or a television, or computing applications, such as computergames, multimedia applications, or office applications, via indirectinput devices, including, for example, keyboards, joysticks, or remotecontrollers. Electronics manufacturers have developed systems thatdetect a user’s movements or gestures and cause the display to respondin a contextually relevant manner. The user’s gestures may be detectedusing an optical imaging system, and characterized and interpreted bysuitable computational resources. For example, a user near a TV mayperform a sliding hand gesture, which is detected by thegesture-recognition system; in response to the detected gesture, the TVmay activate and display a control panel on the screen, allowing theuser to make selections thereon using subsequent gestures; for example,the user may move her hand in an “up” or “down” direction, which, again,is detected and interpreted to facilitate channel selection.

A user of a gesture-based system may wish to perform a variety ofdifferent tasks and/or issue a variety of different types of commands. Auser of a traditional system may enter letters with a keyboard, move apointer with a mouse, and activate or select a widget with a mousebutton; the gesture-based user does not have a similar variety ofdifferent input means. Instead, the gesture-based user issues differenttypes of commands with different gestures. A need therefore exists for agesture-based system that switches between input modes in an intuitive,reliable, and easy-to-use manner.

SUMMARY

In various implementations of the disclosed technology a user of asystem controllable with gesture-based input commands intends to switchfrom a first mode of operation, such as a “move” mode, to a second modeof operation, such as an “action,” “draw” or “mouse click” mode. Theintent of a user to switch modes may be determined by analyzing theuser’s gestures. In one implementation, the system analyzes the path ofmotion of user gestures and determines user intent based thereon. Forexample, if the user is drawing on a 2D virtual canvas or piece ofpaper, the system analyzes the user’s strokes and determines whichstrokes are intended to draw on the canvas and which strokes areintended to move to a new section of the canvas without drawing thereon.In one implementation the analysis is based on the curvature of thestrokes relative to the canvas. In another implementation, the systememploys a hysteresis-like effect to switch between modes. In thisimplementation, small or slow gestures are interpreted as user intent tostay within the current mode, while the similar gestures performed witha greater range of motion and/or speed are interpreted as user intent toswitch modes. A physics-based computer model may be used to model thehysteresis and mode-switching.

In one aspect, a gesture-recognition system includes an image-capturedevice for obtaining digital images of an object in 3D space, aprocessor-executable analysis module for computationally determining apath of movement of the object based on the captured images, and acomputer memory for storing a digital representation of the path ofmovement. A processor-executable mode-control module is configured forinterpreting user gestures in accordance with a first mode of operation;analyzing the path of movement to determine an intent of a user tochange modes of operation; and, upon determining an intent of the userto change modes of operation, subsequently interpreting user gestures inaccordance with the second mode of operation.

The system may include a display for displaying the object. Themode-control module may be configured to analyze the path of movement bydetermining an amount of curvature of the path of movement or adirection of curvature of the path of movement and/or configured toanalyze the path of movement by determining whether the path intersectsa virtual reference plane. The mode-control module may be configured tocreate the virtual reference plane based on prior gestures and/orconfigured to determine if a velocity, acceleration, or range of motionof the movement crosses a threshold to overcome a hysteresis effectassociated with the first mode. The hysteresis effect may be based on aphysics-based model of the first and second modes. The mode-controlmodule may be configured to analyze a second path of movement todetermine the intent of the user to perform a multi-object gesture andchange the modes of operation, the mode-control module subsequentlyinterpreting user gestures in accordance with the second mode ofoperation based on the intent. The mode-control module may be furtherconfigured for determining the intent of a user to change modes based onprior gestures.

In another aspect, a method of computationally recognizing andinterpreting gestures includes obtaining, using an image-capture device,digital images of a real-world object in 3D space; computationallydetermining a path of movement of the object based on the capturedimages; storing, in a computer memory, a digital representation of thepath of movement; interpreting user gestures in accordance with a firstmode of operation; analyzing the path of movement to determine an intentof a user to change modes of operation; and upon determining an intentof the user to change modes of operation, subsequently interpreting usergestures in accordance with the second mode of operation.

Analyzing the path of movement may include determining an amount ofcurvature of the path of movement or a direction or orientation ofcurvature of the path of movement and/or determining whether the pathintersects a virtual reference plane. The virtual reference plane may becreated based on prior gestures. Analyzing the path of movement mayinclude determining if a velocity, acceleration, or range of motion ofthe movement crosses a threshold to overcome a hysteresis effectassociated with the first mode. The hysteresis effect may be based on aphysics-based model of the first and second modes. A second path ofmovement may be analyzed to determine the intent of the user to performa multi-object gesture and changing from the first mode to the secondmode based on the intent. The intent of a user to change modes may bedetermined based on prior gestures.

Advantageously, these and other aspects enable machines, computersand/or other types of intelligent devices, and/or other types ofautomata to obtain information about objects, events, actions, and/orusers employing gestures, signals, and/or other motions conveyingmeaning and/or combinations thereof. These and other advantages andfeatures of the implementations herein described, will become moreapparent through reference to the following description, theaccompanying drawings, and the claims. Furthermore, it is to beunderstood that the features of the various implementations describedherein are not mutually exclusive and can exist in various combinationsand permutations.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings, like reference characters generally refer to like partsthroughout the different views. Also, the drawings are not necessarilyto scale, with an emphasis instead generally being placed uponillustrating the principles of the technology disclosed. In thefollowing description, various implementations of the technologydisclosed are described with reference to the following drawings, inwhich:

FIG. 1 is a simplified block diagram of an exemplary task environment inaccordance with implementations of the technology disclosed;

FIG. 2 is a simplified block diagram of an exemplary system fordetecting a user gesture and modifying a cursor based thereon inaccordance with implementations of the technology disclosed;

FIGS. 3A, 3B and 3C illustrate an implementation of mode-switching basedon gesture paths in accordance with implementations of the technologydisclosed; and

FIG. 4 illustrates an implementation of mode-switching based on aphysical model in accordance with implementations of the technologydisclosed.

DETAILED DESCRIPTION

Motion-capture systems generally include a camera for acquiring imagesof an object; a computer for processing the images to identify andcharacterize the object; and a computer display for displayinginformation related to the identified/characterized object. A lightsource may also be included to illuminate the object. FIG. 1 illustratesan exemplary motion-capture system 100. The system 100 includes one ormore light-capturing devices 102 (e.g., digital cameras or similardevices), each including an image sensor (e.g., a CCD or CMOS sensor),an associated imaging optic (e.g., a lens), and a window of transparentmaterial protecting the lens from the environment. Two or more cameras102 may be arranged such that their fields of view overlap in a viewedregion. One or more light-emitting devices 104 may be used to illuminatean object 106 in the field of view. The cameras 102 provide digitalimage data to a computer 108, which analyzes the image data to determinethe 3D position, orientation, and/or motion of the object 106 the fieldof view of the cameras 102. As used herein, the term “object” broadlyconnotes any real-world item. Typically, objects with gestural relevancemay include user’s finger, hand or other body part, or an item held by auser in performing a gesture, or in some cases, the user herself.

The cameras 102 may include visible-light cameras, infrared (IR)cameras, ultraviolet cameras, or cameras operating in any otherelectromagnetic frequency regime. Preferably, the cameras 102 arecapable of capturing video images. The particular capabilities ofcameras 102 may vary as to frame rate, image resolution (e.g., pixelsper image), color or intensity resolution (e.g., number of bits ofintensity data per pixel), focal length of lenses, and depth of field.In general, for a particular application, any cameras capable offocusing on objects within a spatial volume of interest can be used. Forinstance, to capture motion of the hand of an otherwise stationaryperson, the volume of interest might be a cube of one meter sides. Tocapture motion of a running person, the volume of interest might havedimensions of tens of meters in order to observe several strides.

The cameras may be oriented in any convenient manner. In oneimplementation, the optical axes of the cameras 102 are parallel, butother orientations of the optical axes are within the scope of thetechnology disclosed. As described below, each camera 102 may be used todefine a “vantage point” from which the object 106 is seen; if thelocation and view direction associated with each vantage point areknown, the locus of points in space that project onto a particularposition in the camera’s image plane may be determined. In someimplementations, motion capture is reliable only for objects in an areawhere the fields of view of cameras 102 overlaps and the cameras 102 maybe arranged to provide overlapping fields of view throughout the areawhere motion of interest is expected to occur. In other implementations,the system 100 may include one or more light sources 104, and thecameras 102 measure the reflection of the light emitted by the lightsources on objects 106. The system may include, for example, two cameras102 and one light source 104; one camera 102 and two light sources 104;or any other appropriate combination of light sources 104 and cameras102.

Computer 108 may generally be any device or combination of devicescapable of processing image data using techniques described herein. FIG.2 is a simplified block diagram of a suitably programmed general-purposecomputer 200 implementing the computer 108 according to animplementation of the technology disclosed. The computer 200 includes aprocessor 202 with one or more central processing units (CPUs), volatileand/or non-volatile main memory 204 (e.g., RAM, ROM, or flash memory),one or more mass storage devices 206 (e.g., hard disks, or removablemedia such as CDs, DVDs, USB flash drives, etc. and associated mediadrivers), a display device 208 (e.g., a liquid crystal display (LCD)monitor), user input devices such as keyboard 210 and mouse 212, and oneor more buses 214 (e.g., a single system bus shared between allcomponents, or separate memory and peripheral buses) that facilitatecommunication between these components.

The cameras 102 and/or light sources 104 may connect to the computer 200via a universal serial bus (USB), FireWire, or other cable, orwirelessly via Bluetooth, Wi-Fi, etc. The computer 200 may include acamera interface 216, implemented in hardware (e.g., as part of a USBport) and/or software (e.g., executed by processor 202), that enablescommunication with the cameras 102 and/or light sources 104. The camerainterface 216 may include one or more data ports and associated imagebuffers for receiving the image frames from the cameras 102; hardwareand/or software signal processors to modify the image data (e.g., toreduce noise or reformat data) prior to providing it as input to amotion-capture or other image-processing program; and/or control signalports for transmit signals to the cameras 102, e.g., to activate ordeactivate the cameras, to control camera settings (frame rate, imagequality, sensitivity, etc.), or the like.

The main memory 204 may be used to store instructions to be executed bythe processor 202, conceptually illustrated as a group of modules. Thesemodules generally include an operating system (e.g., a MicrosoftWINDOWS, Linux, or APPLE OS X operating system) that directs theexecution of low-level, basic system functions (such as memoryallocation, file management, and the operation of mass storage devices),as well as higher-level software applications such as, e.g., amotion-capture (mocap) program 218 for analyzing the camera images totrack the position of an object of interest and/or a motion-responseprogram for computing a series of output images (or another kind ofresponse) based on the tracked motion. Suitable algorithms formotion-capture program are described further below as well as, in moredetail, in U.S. Pat. Application Nos. 13/414,485, filed on Mar. 7, 2012and 13/742,953, filed on Jan. 16, 2013, and U.S. Provisional Pat.Application No. 61/724,091, filed on Nov. 8, 2012, which are herebyincorporated herein by reference in their entirety. The various modulesmay be programmed in any suitable programming language, including,without limitation high-level languages such as C, C++, C#, OpenGL, Ada,Basic, Cobra, Fortran, Java, Lisp, Perl, Python, Ruby, or Object Pascal,or low-level assembly languages.

The memory 204 may further store input and/or output data associatedwith execution of the instructions (including, e.g., input and outputimage data 220) as well as additional information used by the varioussoftware applications; for example, in some implementations, the memory204 stores an object library 222 of canonical models of various objectsof interest. As described below, a gesture-recognition module 224 maydetect an object in the camera images that may be identified by matchingits shape to a model in the object library 222, and the model may theninform further image analysis, motion prediction, etc.

In various implementations, the motion captured in a series of cameraimages is used to compute a corresponding series of output images fordisplay on the computer screen 208. For example, camera images of amoving hand may be translated into a wire-frame or other graphicdepiction of the hand by the processor 202. Alternatively, hand gesturesmay be interpreted as input used to control a separate visual output; byway of illustration, a user may be able to use upward or downwardswiping gestures to “scroll” a webpage or other document currentlydisplayed, or open and close her hand to zoom in and out of the page. Inany case, the output images are generally stored in the form of pixeldata in a frame buffer, which may, but need not be, implemented in mainmemory 204. A video display controller reads out the frame buffer togenerate a data stream and associated control signals to output theimages to the display 208. The video display controller may be providedalong with the processor 202 and memory 204 on-board the motherboard ofthe computer 200, and may be integrated with the processor 202 orimplemented as a co-processor that manipulates a separate video memory.In some implementations, the computer 200 is equipped with a separategraphics or video card that aids with generating the feed of outputimages for the display 208. The video card generally includes agraphical processing unit (“GPU”) and video memory, and is useful, inparticular, for complex and computationally expensive image processingand rendering. The graphics card may implement the frame buffer and thefunctionality of the video display controller (and the on-board videodisplay controller may be disabled). In general, the image-processingand motion-capture functionality of the system may be distributedbetween the GPU and the main processor 202 in various conventional waysthat are well characterized in the art.

The computer 200 is an illustrative example; variations andmodifications are possible. Computers may be implemented in a variety ofform factors, including server systems, desktop systems, laptop systems,tablets, smart phones or personal digital assistants, and so on. Aparticular implementation may include other functionality not describedherein, e.g., wired and/or wireless network interfaces, media playingand/or recording capability, etc. In some implementations, one or morecameras may be built into the computer rather than being supplied asseparate components. Further, the computer processor may be ageneral-purpose microprocessor, but depending on implementation canalternatively be, e.g., a microcontroller, peripheral integrated circuitelement, a customer-specific integrated circuit (“CSIC”), anapplication-specific integrated circuit (“ASIC”), a logic circuit, adigital signal processor (“DSP”), a programmable logic device such as afield-programmable gate array (“FPGA”), a programmable logic device(“PLD”), a programmable logic array (“PLA”), smart chip, or other deviceor arrangement of devices.

Further, while computer 200 is described herein with reference toparticular blocks, this is not intended to limit the technologydisclosed to a particular physical arrangement of distinct componentparts. For example, in some implementations, the cameras 102 areconnected to or integrated with a special-purpose processing unit that,in turn, communicates with a general-purpose computer, e.g., via directmemory access (“DMA”). The processing unit may include one or more imagebuffers for storing the image data read out from the camera sensors, aGPU or other processor and associated memory implementing at least partof the motion-capture algorithm, and a DMA controller. The processingunit may provide processed images or other data derived from the cameraimages to the computer for further processing. In some implementations,the processing unit sends display control signals generated based on thecaptured motion (e.g., of a user’s hand) to the computer, and thecomputer uses these control signals to adjust the on-screen display ofdocuments and images that are otherwise unrelated to the camera images(e.g., text documents or maps) by, for example, shifting or rotating theimages.

In one implementation, a cursor-mode module 226 maintains differentmodes for an on-screen cursor, or other interactive on-screen object,that is controlled by user gestures; the cursor-mode module 226 switchesbetween the modes by analyzing the gestures to determine the intent of auser to switch modes. Typical scenarios in which a user may intend toswitch between modes include first moving an on-screen cursor and thenclicking with that cursor and first drawing a stroke of a letter,number, or any other symbol or object and then moving the cursor toanother location without drawing. The disclosed technology is notlimited, however, to any particular type of modes, and theimplementations described herein may be applied to any mode in anyapplication, game, or other computer program or program interface.

The on-screen drawing of the capital letter H using gestures is hereinpresented as an example using an implementation of the technologydisclosed. One of skill in the art will understand, however, that thetechniques and systems described are not limited to only this or otherletters and may be applied to a variety of applications and uses, suchas drawing, painting, scrolling, pressing virtual buttons, manipulatingvirtual scrollbars or sliders, or providing input to games or otherapplications. FIG. 3A illustrates a system 300 including a computerdisplay 302 and a user’s hand 304. The display may be the display 208illustrated in FIG. 2 ; some or all of the remainder of the system ofFIG. 2 may also be included in the system 300 of FIGS. 3 , but is notpictured for clarity. The processor 202 is executing, in thisimplementation, a drawing or text-entry program for interpretinggestures made by the hand 304 as strokes of characters. The hand 304performs a series of gestures, including a first drawing gesture 306A(which the system 300 interprets, using the gesture-recognition module224 and the systems and methods described above and the cursor-modemodule 226 described below) as a first downstoke 306B of an on-screen H.The user next makes a first movement gesture 308 to set up a seconddrawing gesture 310A; the system does not draw anything on the screen302 corresponding to the first movement gesture 308 but does draw asecond on-screen downstroke 310B for the second drawing gesture 310A.The user makes a second movement gesture 312 to set up a third drawinggesture 314A to draw the on-screen cross-stroke 314B.

At each transition between a drawing gesture 306A, 310A, 314A and amovement gesture 308, 312, the cursor-mode module 226 detects a userintent to switch between drawing and movement modes. In oneimplementation, the cursor-mode module 226 analyzes the amount andorientation of curvature present in the path that the hand 304 makeswhile performing the gestures. For example, FIG. 3B illustrates the usergestures in this implementation from a perspective horizontallyperpendicular to the perspective shown FIG. 3A. In this example, thecursor-mode module 226 may detect that the drawing gestures 306A, 310A,314A have little curvature or are straight or nearly straight and themovement gestures 308, 312 are curved. In this example, the movement ofthe hand 304 or a point on the hand 304, such as the tip of its indexfinger 316, follows a straighter path through 3D space when making thedrawing gestures 306A, 310A, 314A and a more curved path through 3Dspace when making the movement gestures 308, 312. In anotherimplementation, the orientation of curvature is used to determine theintended mode. FIG. 3C illustrates the user gestures in thisimplementation from a perspective horizontally perpendicular to theperspective shown FIG. 3A. The path drawn through 3D space by the hand304 may be curved in a first direction when making the drawing gestures306A, 310A, 314A (e.g., the midpoint of the curved path is closer to thescreen 302 than the endpoints of the curved path) and curved in a seconddirection when making the movement gestures 308, 312 (e.g., themidpoints of these curved paths are farther away from the screen 302than the endpoints). In another implementation the amount of curvatureis use to determine the intended mode. For example, a threshold ofcurvature may be used to separate a drawing gesture from a movementgesture. The amount of curvature may be determined by the maximumdistance between the path of the curve and an ideal straight linebetween the endpoints of the gesture. An amount of curvature above athreshold of curvature may indicate intent to be in the first mode andbelow intent to be in the second mode. The threshold between a firstmode and second mode may be a fixed distance (e.g., one to fivecentimeters). In another implementation, the cursor-mode module 226models the path of the gesture as a mathematical curve (e.g., a parabolaor elliptical curve) and determines the mode of the gesture based onparameters of the modeled curve (e.g., the axes of the ellipse orcoefficients of the parabola).

In one implementation, the mode of a gesture is determined when thegesture is completed. In other implementations, the amount and nature ofcurvature of the paths of the gestures 306A, 308, 310A, 312, 314A may bedetermined as the hand 304 begins to trace the paths (i.e., before thegesture is fully completed). For example, the cursor-mode module 226 maydecide the mode of operation of the gestures 306A, 308, 310A, 312, 314Aafter the hand 304 has moved a given distance (e.g., five or tencentimeters) or has completed a certain percentage of a predicted totalpath distance (e.g., five or ten percent). In another implementation,the cursor-mode module 226 decides the mode of operation of the gestures306A, 308, 310A, 312, 314A after a given amount of curvature orstraightness has been detected (e.g., once a path has deviated by one ortwo centimeters or five or ten degrees away from a virtual referenceplane or line).

The virtual reference plane, or other reference use for determining anamount of curvature, may be a vertical plane, or other fixedpredetermined plane, and/or may be determined by previous gestures. Forexample, the plane defined by the first drawing gesture 306A may be usedto determine whether the first movement gesture 308 is straight orcurved. Additional drawing gestures made by the hand may be used tofurther define the virtual reference plane; in one implementation, anaverage of all, or a fixed running average of some, drawing gestures areused to define the reference plane. In another implementation, gesturesmade in which the index finger 316 occupies the same space as thevirtual reference plane are deemed to be drawing gestures and gesturesmade in which the index finger 316 is separated by some distance (e.g.,one to five centimeters) from the virtual reference plane at some pointduring the gesture (e.g., the midpoint) are deemed to be movementgestures.

In one implementation, the cursor-mode module 226 uses the direction inwhich an object is pointed to determine the mode of operation of thegestures. For example, the direction the index finger 316 is pointingmay be used to determine the mode of operation of the gestures 306A,308, 310A, 312, 314A. If the index finger 316 is pointing in a firstdirection, for example toward a point located above a horizontal plane,when it begins a gesture, the cursor-mode module 226 determines the modeof operation is a drawing mode. In this same example, if the indexfinger 316 is pointing in a second direction, for example toward a pointlocated below the horizontal plane, when it begins a gesture, thecursor-mode module 226 may determine the mode of operation is a movementmode. In another implementation, the cursor-mode module 226 switches todrawing mode for downward (or mostly downward) gestures and to movementmode for upward (or mostly upward) gestures.

When determining mode the cursor-mode module 226 is not limited to usinga single factor and may consider any combination of the disclosedfactors including; amount of path curvature, curvaturedirection/orientation, distance from reference plane, pointingdirection, and gesture direction. In implementations using multiplefactors to determine mode, factors may be weighted to more heavilyinfluence the mode determination. Further, when determining mode thecursor-mode module 226 is not limited determining between two modes andmay use any combination to the factors to determine the mode ofoperation from more than two modes of operation.

As described above, the gesture-recognition module 224 and/or thecursor-mode module 226 may not determine a mode for a gesture untilafter the object making the gesture has traveled a certain distance in3D space. In implementations, certain applications, such as the drawingapplication discussed above with reference to FIGS. 3 , may display adiscontinuity or jolt to a user as this traveled distance is suddenlyfilled in by the application once the mode is determined to be a drawingmode. In an implementation, instead of filling in the discontinuity allat once, the drawing application re-traces the already traveled path ofthe object, drawing in the path at a higher velocity than that traveledby the object. For example, if the object is moving at 10 cm/s and ithas already moved 5 cm before the cursor-mode module 226 determined thatthe movement is a drawing gesture, the 0-5 cm distance is filled in from0 to 5 cm at a rate of 20 cm/s until the filled-in line “catches up” tothe object, at which point the drawing rate drops to the object-movementrate. In one implementation, the gesture-recognition module 224 and/orthe cursor-mode module 226 pass position data relating to the object tothe drawing application (or any other application) to facilitate thisdrawing method; for example, no position data is passed until thecursor-mode module 226 determines the mode of the gesture, after whichthe faster-rate motion is passed until the drawing catches up to thecurrent position of the object, after which normal-rate positioninformation is passed.

In other implementations, the gesture-recognition module 224 and/or thecursor-mode module 226 “guess” a mode of operation before the actualmode of operation is determined. The guess may be to always first assumea drawing mode or a movement mode. In either or both cases, any lines orother markings displayed on the screen 302 during the guessing periodmay be displayed in a color or shade different from the other lines ormarkings (e.g., grey instead of black). If the actual mode of operationis drawing the displayed different color becomes the intended color andif the actual mode of operation is movement the displayed differentcolor disappears. In other implementations, the guess may be made basedon prior gestures. For example, if the last gesture was a drawinggesture the current gesture may be guessed as movement, and vice versa.

The cursor-mode module 226 may switch modes of a cursor based onmultiple tracked objects. For example, a user may make a gesture usingtwo or more fingers (e.g., a “pinching” gesture using a thumb and indexfinger or a gesture using an index finger on each hand) or using astylus held in one hand and a finger on the other hand. In theseimplementations, the cursor-mode module 226 may switch to a modeassociated with the particular multi-object gesture even if, forexample, only one object intersects the virtual plane discussed above.

In some implementations of the technology disclosed, the cursor-modemodule 226 implements a physics-based computer model to switch betweenmodes. The model may implement a hysteresis-like effect between themodes. In these implementations, small or slow gestures are interpretedas user intent to stay within the current mode and gestures withsimilarly shaped paths performed with a range of motion and/or velocitygreater than a threshold are interpreted as user intent to switch modes.In one implementation, the two modes are a cursor-movement mode and aclicking/interaction mode. In each mode, the gesture-recognition module224 is more sensitive to gestures corresponding to that mode and lesssensitive to gestures that do not correspond to that mode. For example,while in the clicking mode, the gesture-recognition module 224 mayregister button-pressing motions as clicks even if the button-pressingmotions are made using only a user’s finger, are made slowly, or aremade by the user’s finger traveling only a short distance. In order toenter the clicking mode, however, the cursor-mode module 226 may requirebutton-pressing motions made with a user’s entire hand (or with morethan one finger), faster motions, and/or motions made over a greaterdistance. Motions that the cursor-mode module 226 registers as clickswhile in the clicking mode may not be sufficient to enter the clickingmode.

FIG. 4 illustrates a physics-based computer model 400 that thecursor-mode module 226 may implement to switch modes. The model includesa virtual “hill” 402 having a first “valley” 404 on one side and asecond “valley” 406 on its other side. A virtual “ball” 408A, 408Bexists in either the first 404 or second 406 valley. One valley may beassociated with a first mode of operation (e.g., movement) while theother valley may be associated with a second mode of operation (e.g.,clicking). The intensity, distance, velocity, and/or acceleration of auser gesture “pushes” the ball 408A, 408B up the hill 402. Less-intensegestures do not push the ball 408A, 408B with enough force to surmountthe hill 404 and enter the other valley 404, 406; only a more intensegesture may do so, and thereby switch modes. For example, if the ball408A is in the first valley 404, the cursor-mode module 226 selects amovement mode; in order to switch to a clicking mode associated with thesecond valley 406, the user must make a clicking motion of greatervelocity, acceleration, or distance to enter the clicking mode thanwould otherwise be required, while in the clicking mode, to create aclick.

Implementations may be employed in a variety of application areas, suchas for example and without limitation consumer applications includinginterfaces for computer systems, laptops, tablets, television, gameconsoles, set top boxes, telephone devices and/or interfaces to otherdevices; medical applications including controlling devices forperforming robotic surgery, medical imaging systems and applicationssuch as CT, ultrasound, x-ray, MRI or the like, laboratory test anddiagnostics systems and/or nuclear medicine devices and systems;prosthetics applications including interfaces to devices providingassistance to persons under handicap, disability, recovering fromsurgery, and/or other infirmity; defense applications includinginterfaces to aircraft operational controls, navigations systemscontrol, on-board entertainment systems control and/or environmentalsystems control; automotive applications including interfaces toautomobile operational systems control, navigation systems control,on-board entertainment systems control and/or environmental systemscontrol; security applications including, monitoring secure areas forsuspicious activity or unauthorized personnel; manufacturing and/orprocess applications including interfaces to assembly robots, automatedtest apparatus, work conveyance devices such as conveyors, and/or otherfactory floor systems and devices, genetic sequencing machines,semiconductor fabrication related machinery, chemical process machineryand/or the like; and/or combinations thereof.

Implementations of the technology disclosed may further be mounted onautomobiles or other mobile platforms to provide information to systemstherein as to the outside environment (e.g., the positions of otherautomobiles). Further implementations of the technology disclosed may beused to track the motion of objects in a field of view or used inconjunction with other mobile-tracking systems. Object tracking may beemployed, for example, to recognize gestures or to allow the user tointeract with a computationally rendered environment; see, e.g., U.S.Pat. Application Serial Nos. 61/752,725 (filed on Jan. 15, 2013) and13/742,953 (filed on Jan. 16, 2013), the entire disclosures of which arehereby incorporated by reference.

It should also be noted that implementations of the technology disclosedmay be provided as one or more computer-readable programs embodied on orin one or more articles of manufacture. The article of manufacture maybe any suitable hardware apparatus, such as, for example, a floppy disk,a hard disk, a CD ROM, a CD-RW, a CD-R, a DVD ROM, a DVD-RW, a DVD-R, aflash memory card, a PROM, a RAM, a ROM, or a magnetic tape. In general,the computer-readable programs may be implemented in any programminglanguage. Some examples of languages that may be used include C, C++, orJAVA. The software programs may be further translated into machinelanguage or virtual machine instructions and stored in a program file inthat form. The program file may then be stored on or in one or more ofthe articles of manufacture.

Certain implementations of the technology disclosed were describedabove. It is, however, expressly noted that the technology disclosed isnot limited to those implementations, but rather the intention is thatadditions and modifications to what was expressly described herein arealso included within the scope of the technology disclosed. For example,it may be appreciated that the techniques, devices and systems describedherein with reference to examples employing light waves are equallyapplicable to methods and systems employing other types of radiantenergy waves, such as acoustical energy or the like. Moreover, it is tobe understood that the features of the various implementations describedherein were not mutually exclusive and can exist in various combinationsand permutations, even if such combinations or permutations were notmade express herein, without departing from the spirit and scope of thetechnology disclosed. In fact, variations, modifications, and otherimplementations of what was described herein will occur to those ofordinary skill in the art without departing from the spirit and thescope of the technology disclosed. As such, the disclosed technology isnot to be defined only by the preceding illustrative description.

What is claimed is:
 1. A system for gesture-recognition, comprising: acomputer memory for storing a digital representation of a determinedpath of movement of an object of interest in a three-dimensional (3D)space; and a processor coupled to the computer memory; and aprocessor-executable mode-control module configured to: display a linefollowing movement of the object of interest; and fill in a displayedline at a rate that is faster than a detected rate of the object ofinterest until the displayed line catches up with the object ofinterest, and thereafter displaying movement of the object of interestat the detected rate of the object of interest.
 2. The system of claim1, wherein the processor-executable mode-control module is furtherconfigured to: obtain an intent of a user to change from another mode ofoperation to a first mode of operation comprising a drawing mode basedat least in part upon a determined pointing direction of the object ofinterest; and perform filling in of the displayed line when in thedrawing mode.
 3. The system of claim 2, wherein filling in of thedisplayed line includes passing position data relating to the object ofinterest to a drawing application.
 4. The system of claim 3, wherein noposition data is passed until a mode is determined, after whichfaster-rate motion is passed until the displayed line catches up to acurrent position of the object of interest.
 5. The system of claim 4,further including passing normal-rate position information after thedisplayed line catches up to the current position of the object ofinterest.
 6. The system of claim 2, wherein a discontinuity or jolt isdisplayed as distance traveled by the object of interest is filled inonce a mode of operation is determined to be the drawing mode.
 7. Thesystem of claim 1, further including re-tracing a path already traveledby the object of interest by drawing in the path at a higher velocitythan that traveled by the object of interest.
 8. The system of claim 2,wherein a mode of operation for a gesture is not determined until afterthe object of interest making the gesture has traveled some distance in3D space.
 9. The system of claim 2, wherein no line is displayed formovement along a path when a mode of operation is a movement mode.
 10. Amethod of computationally interpreting gestures, the method comprising:displaying a line following movement of an object of interest along apath of movement of the object of interest in a three-dimensional (3D)space; and filling in a displayed line at a rate that is faster than adetected rate of the object of interest until the displayed line catchesup with the object of interest, and thereafter displaying movement ofthe object of interest at the detected rate of the object of interest.11. The method of claim 10, further comprising: obtaining an intent of auser to change from another mode of operation to a first mode ofoperation comprising a drawing mode based at least in part upon adetermined pointing direction of the object of interest; and performingthe filling in of the displayed line when in the drawing mode.
 12. Themethod of claim 11, wherein the filling in of the displayed lineincludes passing position data relating to the object of interest to adrawing application.
 13. The method of claim 12, wherein no positiondata is passed until a mode is determined, after which faster-ratemotion is passed until the displayed line catches up to a currentposition of the object of interest.
 14. The method of claim 13, furtherincluding passing normal-rate position information after the displayedline catches up to the current position of the object of interest. 15.The method of claim 11, further including displaying a discontinuity orjolt as distance traveled by the object of interest is filled in once amode of operation is determined to be the drawing mode.
 16. The methodof claim 10, further including re-tracing a path already traveled by theobject of interest by drawing in the path at a higher velocity than thattraveled by the object of interest.
 17. The method of claim 11, whereina mode of operation for a gesture is not determined until after theobject of interest making the gesture has traveled some distance in 3Dspace.
 18. The method of claim 11, wherein no line is displayed formovement along a path when a mode of operation is a movement mode.
 19. Anon-transitory computer readable storage medium impressed with computerprogram instructions to computationally interpret gestures, whichinstructions, when executed on a processor, implement: displaying a linefollowing movement of an object of interest along a path of movement ofthe object of interest in a three-dimensional (3D) space; and filling ina displayed line at a rate that is faster than a detected rate of theobject of interest until the displayed line catches up with the objectof interest, and thereafter displaying movement of the object ofinterest at the detected rate of the object of interest.
 20. Thenon-transitory computer readable storage medium of claim 19, wherein theinstructions, when executed on the processor, further implement:obtaining an intent of a user is change from another mode of operationto a first mode of operation comprising a drawing mode based at least inpart upon a determined pointing direction of the object of interest; andperforming the filling in of the displayed line when in the drawingmode.